- NSF BCS-1238599
- NICHD U01-HD-076595
- Society for Research in Child Development (SRCD)
- NYU Libraries, NYU IT, IRB, and OSP
Acknowledgments
Is psychological science reproducible?
- Nosek et al. (Collaboration 2015)
- vs. Gilbert et al. (Gilbert et al. 2016)
- Sharing data focuses attention on substantive issues
Databrary.org
- Digital data library specialized for research video
- Video/audio + participant/context metadata
- Restricted access for research/educational use
- Policy framework for sharing identifiable data
Video uniquely captures behavior
Video uniquely captures behavior
https://nyu.databrary.org/slot/9840/-/asset/11193/download?inline=true
(Frank 2014)
Unequivocally & unambiguously demonstrates phenomena
Video depicts unreported methodological details (Adolph 2013)
Datavyu makes video coding reproducible
- Raw research video must be coded
- Datavyu.org = free, open source coding tool
- Add codes, annotations time-locked to video segments
- Ruby API for scripting reproducible workflows
Datavyu facilitates reproducible video coding
Databrary enhances reproducibility
- "Active" curation of data as it is collected
- Organize, share, standardized participant metadata
- Sharing based on
- user access level
- participant permission
(Adolph 2013)
Standardized – reproducible – release levels
Databrary enhances reproducibility
- Share raw research videos
- Share video excerpts for teaching and learning
- Share materials, coding spreadsheets, displays
- Share links to papers, code repositories
- Shared datasets get DOIs
- Long-term preservation via partnership with NYU Libraries
Take homes
- Video uniquely captures complexity & richness of behavior
- Databrary securely stores, shares video + metadata
- Databrary and Datavyu enhance reproducibility of video-based research
- Databrary makes psychological science more transparent & reproducible
- http://github.com/databrary/presentations/nyu-data-science-reproducibility-16
To boldly go where no science has gone before…
- Reproducibility of numerical computation ≠
- Reproducibility of behavioral measures, manipulations
- All behavioral researchers should
- share reproducible workflows of data, analyses AND
- videos of behavioral measures, displays
Join us!
References
Adolph, Karen; 2013. “Infants Crawling and Walking over High and Low Bridges.” Databrary. doi:10.17910/B7MW2K.
Collaboration, Open Science. 2015. “Estimating the Reproducibility of Psychological.” Science 349 (6251): aac4716. doi:10.1126/science.aac4716.
DeLoache, Judy. 2014. “Scale Errors Offer Evidence for a Perception-Action Dissociation Early in Life.” Databrary. doi:10.17910/B7H019.
DeLoache, Judy S., David H. Uttal, and Karl S. Rosengren. 2004. “Scale Errors Offer Evidence for a Perception-Action Dissociation Early in Life.” Science 304 (5673): 1027–9. doi:10.1126/science.1093567.
Frank, Michael C.; 2014. “Representing Exact Number Visually Using Mental Abacus.” Databrary. doi:10.17910/B7PP4W.
Gilbert, Daniel T., Gary King, Stephen Pettigrew, and Timothy D. Wilson. 2016. “Comment on ‘Estimating the Reproducibility of Psychological Science’.” Science 351 (6277): 1037–7. doi:10.1126/science.aad7243.